Methods, systems, and media for presenting supplemental information corresponding to on-demand media content

ABSTRACT

Methods, systems, and media for presenting supplemental information corresponding to on-demand media content are provided. In some implementations, the method comprises: receiving an indication from a web browser extension executing within a web browsing application that a media content item is being presented on a content provider website; identifying the media content item being presented on the content provider website; determining whether supplemental content associated with the broadcasted media content item is available; in response to determining that the supplemental content is available, causing the web browser extension to present an indicator that the supplemental content is available; obtaining the supplemental content corresponding to the broadcasted media content item, wherein each piece of the supplemental content is associated with timing information of the broadcasted media content item; and causing a portion of the obtained supplemental content to be presented concurrently with the media content item in response to detecting that the indicator has been selected.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/005,048, filed Jun. 11, 2018, which is a continuation of U.S. patentapplication Ser. No. 15/649,370 filed Jul. 13, 2017, which is acontinuation of U.S. patent application Ser. No. 15/272,027, filed Sep.21, 2016, which is a continuation of U.S. patent application Ser. No.14/191,059, filed Feb. 26, 2014, which claims the benefit of U.S.Provisional Patent Application No. 61/922,236, filed Dec. 31, 2013, eachof which is hereby incorporated by reference herein in its entirety.

TECHNICAL FIELD

The disclosed subject matter relates to methods, systems, and media forpresenting supplemental information corresponding to on-demand mediacontent.

BACKGROUND

While watching a program, a viewer is often interested in informationrelating to the program. For example, the viewer may want to search foradditional information about a topic that is discussed in the program.

Search engines allow the viewer to search on the Internet for resources,which include webpages, images, video content, and audio content. Atypical search engine provides a web page for entering search terms,where the search terms are inputted by the viewer. Search resultsconducted for particular search terms are provided to the viewer.Moreover, with the advent of mobile devices, the use of mobile searchengines has increased. For example, using a mobile device, the viewermay perform a search and browse through Internet content while watchingthe program.

Text-based search engines, however, may be somewhat limited for thisuser. In an example where the user wants to find an article similar tothat which is being discussed during a program, the user may attempt toobtain this information by entering one or more keywords into a searchengine. The user then scans through search results to find a webpagecontaining information relating to the topic of the discussion. This canbe a time consuming procedure for the user. Moreover, this can cause theuser to miss a substantial portion of the program or cause the user topause the program to obtain such information via this procedure.

Accordingly, mechanisms for presenting supplemental informationcorresponding to on-demand media content are provided.

SUMMARY

Methods, systems, and media for presenting supplemental informationcorresponding to on-demand media content are provided.

In accordance with some implementations of the disclosed subject matter,a method for providing information relating to media content isprovided, the method comprising: receiving, using a hardware processor,an indication from a web browser extension executing within a webbrowsing application that a media content item is being presented on acontent provider website; identifying the media content item beingpresented on the content provider website by determining whether themedia content item presented by the content provider website has alsobeen previously broadcasted by a television provider and associating themedia content item with a broadcasted media content item provided by thetelevision provider; determining whether supplemental content associatedwith the broadcasted media content item is available; in response todetermining that the supplemental content is available, causing the webbrowser extension to present an indicator that the supplemental contentis available; obtaining the supplemental content corresponding to thebroadcasted media content item, wherein each piece of the supplementalcontent is associated with timing information of the broadcasted mediacontent item; and causing a portion of the obtained supplemental contentto be presented concurrently with the media content item in response todetecting that the indicator has been selected, wherein the portion ofthe obtain supplemental content is selected based on current timinginformation of the media content item currently being provided by thecontent provider.

In accordance with some implementations of the disclosed subject matter,a system for providing information relating to media content isprovided, the system comprising a hardware processor that is configuredto: receive, from a web browser extension executing within a webbrowsing application, an indication that a media content item is beingpresented on a content provider website; identify the media content itembeing presented on the content provider website by determining whetherthe media content item presented by the content provider website hasalso been previously broadcasted by a television provider andassociating the media content item with a broadcasted media content itemprovided by the television provider; determine whether supplementalcontent associated with the broadcasted media content item is available;in response to determining that the supplemental content is available,cause the web browser extension to present an indicator that thesupplemental content is available; obtain the supplemental contentcorresponding to the broadcasted media content item, wherein each pieceof the supplemental content is associated with timing information of thebroadcasted media content item; and cause a portion of the obtainedsupplemental content to be presented concurrently with the media contentitem in response to detecting that the indicator has been selected,wherein the portion of the obtain supplemental content is selected basedon current timing information of the media content item currently beingprovided by the content provider.

In accordance with some implementations of the disclosed subject matter,a non-transitory computer-readable medium containing computer executableinstructions that, when executed by a processor, cause the processor toperform a method for providing information relating to media content isprovided. The method comprising: receiving an indication from a webbrowser extension executing within a web browsing application that amedia content item is being presented on a content provider website;identifying the media content item being presented on the contentprovider website by determining whether the media content item presentedby the content provider website has also been previously broadcasted bya television provider and associating the media content item with abroadcasted media content item provided by the television provider;determining whether supplemental content associated with the broadcastedmedia content item is available; in response to determining that thesupplemental content is available, causing the web browser extension topresent an indicator that the supplemental content is available;obtaining the supplemental content corresponding to the broadcastedmedia content item, wherein each piece of the supplemental content isassociated with timing information of the broadcasted media contentitem; and causing a portion of the obtained supplemental content to bepresented concurrently with the media content item in response todetecting that the indicator has been selected, wherein the portion ofthe obtain supplemental content is selected based on current timinginformation of the media content item currently being provided by thecontent provider.

In accordance with some implementations of the disclosed subject matter,a system for providing information relating to media content isprovided, the system comprising: means for receiving an indication froma web browser extension executing within a web browsing application thata media content item is being presented on a content provider website;means for identifying the media content item being presented on thecontent provider website by determining whether the media content itempresented by the content provider website has also been previouslybroadcasted by a television provider and associating the media contentitem with a broadcasted media content item provided by the televisionprovider; means for determining whether supplemental content associatedwith the broadcasted media content item is available; in response todetermining that the supplemental content is available, means forcausing the web browser extension to present an indicator that thesupplemental content is available; means for obtaining the supplementalcontent corresponding to the broadcasted media content item, whereineach piece of the supplemental content is associated with timinginformation of the broadcasted media content item; and means for causinga portion of the obtained supplemental content to be presentedconcurrently with the media content item in response to detecting thatthe indicator has been selected, wherein the portion of the obtainsupplemental content is selected based on current timing information ofthe media content item currently being provided by the content provider.

In some implementations, the system further comprises means forcomparing media content information received from the web browserextension with guidance information from the television provider.

In some implementations, the system further comprises means fordetermining whether a uniform resource locator associated with thecontent provider website is included on a list of uniform resourcelocators.

In some implementations, the system further comprises: means forextracting media information from the media content item being presentedon the content provider website; means for generating a fingerprint fromat least a portion of the extracted media information that correspondsto a portion of the media content item; and means for determiningwhether the fingerprint of the portion of the media content item beingpresented on the content provider website matches one of a plurality offingerprints associated with the media content item from an internalvideo database, wherein each of the plurality of fingerprints has atimestamp associated with a portion of the media content item.

In some implementations, the supplemental content is retrieved from adata source that is indexed by at least the media content item andtiming information.

In some implementations, the indicator is presented within an addressbar of the web browsing application.

In some implementations, the system further comprises: means fordetermining whether an updated fingerprint of the media content itembeing presented on the content provider website matches one of aplurality of fingerprints associated with the media content item from aninternal video database; and means for determining a timestampassociated with the media content item.

In some implementations, the system further comprises means fordetermining which of the obtained supplemental content to transmit tothe web browser extension based on the timestamp.

In some implementations, the system further comprises: means forreceiving the current timing information of the media content item froma media player on the content provider website; and means fordetermining which of the obtained supplemental content to transmit tothe web browser extension based on the current timing information.

BRIEF DESCRIPTION OF THE DRAWINGS

Various objects, features, and advantages of the disclosed subjectmatter can be more fully appreciated with reference to the followingdetailed description of the disclosed subject matter when considered inconnection with the following drawings, in which like reference numeralsidentify like elements.

FIG. 1 shows an illustrative example of a content provider website thatprovides media content, such as a television program, where supplementalcontent that relates to the television program is provided through a webbrowser extension in accordance with some implementations of thedisclosed subject matter.

FIG. 2 shows an illustrative example of a process for selecting mediacontent for playback on a content provider website and presentingsupplemental content that is contextually relevant to the presentedmedia content in accordance with some implementations of the disclosedsubject matter.

FIG. 3 shows an illustrative example of a process for determining mediacontent presented by a content provider website and selectingsupplemental content that is contextually relevant to the presentedmedia content in accordance with some implementations of the disclosedsubject matter.

FIG. 4 shows a schematic diagram of an illustrative system suitable forimplementation of the mechanisms described herein for providingsupplemental content, such as text snippets and URLs to content relatedto the context of the media content, in accordance with someimplementations of the disclosed subject matter.

FIG. 5 shows a detailed example of a server and one of the computingdevices of FIG. 4 that can be used in accordance with someimplementations of the disclosed subject matter.

DETAILED DESCRIPTION

In accordance with various implementations, as described in more detailbelow, mechanisms, which can include methods, systems, andcomputer-readable media, for presenting supplemental informationcorresponding to on-demand media content are provided.

In some implementations, the supplemental information presented for anon-demand media program, such as an on-demand television programpresented on a streaming media website, can be based on informationextracted from a corresponding program that has been broadcasted by atelevision provider.

In accordance with some implementations, the mechanisms can includedetermining whether to present supplemental information. For example,the mechanisms can be implemented as a web browser extension executingon a web browsing application that accesses the streaming media website,where the web browser extension processes the webpage providing theon-demand media program, identifies the on-demand media program selectedfor play back, and determines whether supplemental informationassociated with the on-demand media program is available. In a moreparticular example, the mechanisms can determine whether the identifiedon-demand media program is associated with a program that wasbroadcasted by a television program (e.g., by associating an episode ofa television program provided by an on-demand content provider with thesame episode of the television program that was provided by a televisionprovider), where the television program has supplemental contentassociated with it. In response to determining that such supplementalinformation is available, the mechanisms can obtain the supplementalinformation from one or more data sources relating to the programprovided by the television provider and provide an availabilityindicator using the web browser extension.

It should be noted that the supplemental information can include anysuitable information relating to the program. For example, supplementalinformation can include extracted keywords mentioned in subtitleinformation of the program, links to articles relating to one or more ofthe extracted keywords, text snippets of documents relating to one ormore of the extracted keywords, selectable links to on-screen text(e.g., uniform resource locators (URLs), phone numbers, hashtags, etc.),identified product logos, faces, and/or famous places and links torelated documents, identified songs and links to sources for listeningand/or purchasing such songs, identified topics of discussion and linksto news articles or other suitable documents discussing the identifiedtopic, etc.

In some implementations, the mechanisms can present the supplementalinformation in response to receiving an indication that the availabilityindicator has been selected. For example, the mechanisms can present thesupplemental information relating to the on-demand program in a windowdisplayed on the side of the on-demand program as it is being playedback. In another example, the supplemental information can be providedwithin a window that overlays a video window presenting the on-demandprogram. In yet another example, the supplemental information can beprovided as a transparency, where the supplemental information can beoverlaid on a video window presenting the on-demand program withoutinhibiting the viewability of the on-demand program.

In some implementations, supplemental information for on-demand mediacontent can be presented on one or more cards, where a card is a userinterface which can present information to a user. In someimplementations, a user can interact with the card, for example, byindicating that the user viewed and liked the supplemental informationon the card. In another example, a user can cause the card to be removedto indicate a lack of interest in the supplemental information. A cardcan be presented on a user interface on any suitable user device (e.g.,a mobile phone, a tablet computer, a desktop computer, or any othersuitable user device).

In some implementations, the mechanisms can present supplementalinformation that is contextually relevant to the presented on-demandmedia content. For example, the mechanisms can determine the playbackposition of the on-demand media program and present supplementalinformation based on the particular playback position within theon-demand media program. In a more particular example, the mechanismscan enter a synchronization mode, where audio and/or image fingerprintsare extracted and compared with fingerprints of the correspondingprogram that has been broadcast by a television provider. In response,synchronization data can be generated that includes the particularplayback position within the program. Such synchronization data can beused to present particular pieces of contextually relevant informationat particular times during the playback of the on-demand media program.For example, supplemental information relating to the program, such asguidance information, episode information, and general programinformation, can be presented at the start of the program, whilesupplemental information relating to a particular topic can be presentedwhen that particular topic is being discussed within the on-demandprogram.

It should be noted that, although the implementations described hereingenerally relate to providing supplemental information corresponding toon-demand media content, this is merely illustrative. The mechanisms canbe implemented to provide supplemental information with any suitablemedia content provided by an over-the-top (OTT) content source, avideo-on-demand (VOD) content source, streaming media content sources,and/or any other suitable source.

It should also be noted that, although the implementations describedherein generally relate to providing supplemental information based oninformation extracted from a corresponding program that has beenbroadcasted by a television provider, this is merely illustrative.Additionally or alternatively, it can be determined whether an on-demandmedia program is associated with media content from any suitabledatabase of media content. For example, videos from one or more of thesemedia databases can be processed to generate fingerprints and/or extractsupplemental information. In response to determining that the on-demandmedia program presented on a streaming media website corresponds to avideo from an internal media database, the supplemental information canbe provided.

These mechanisms can be used in a variety of applications. For example,these mechanisms can be used to provide supplemental information on oneor more websites that provide on-demand content. In a more particularexample, these mechanisms can be used to determine the context of theposition in the program that a user is currently watching and presentsupplemental information that is relevant to the context of the currentposition in the program. In another more particular example, thesemechanisms can be used to provide an application that presentssupplemental information to users that are consuming video content,where the supplemental information is contextually relevant to the videocontent and where the video content is not available to the application(e.g., content that is provided via a proprietary delivery platform).

Turning to FIG. 1, FIG. 1 shows an example of a web browsing applicationthat, in response to accessing a particular page 100 having a URL 110,displays a program or other suitable media content 120 and supplementalinformation 150, 160, 170, and/or 180 in accordance with someimplementations of the disclosed subject matter. More particularly, asshown in FIG. 1, in response to selecting a particular episode of anon-demand program (e.g., Episode 4 from Season 10 of “The Comedy Show”),a web browser extension executing on the web browsing application canprocess the page 100 providing the on-demand program 120 and determinewhether supplemental information is available to be presented to theuser.

In response to determining that supplemental information is available tothe user, an availability indicator 140 or any other suitable indicationthat indicates supplemental information is available can be provided tothe user. As shown in FIG. 1, availability indicator 140 can be providedwithin the address bar of the web browsing application. Alternatively,availability indicator 140 can be presented in any suitable location,such as a pop-up window, a notification area, etc. In response toselecting availability indicator 140, supplemental informationcorresponding to the on-demand program 120 can be presented to the user.More particularly, supplemental information in the form of cards 150,160, 170, and/or 180 can be presented, where each individual card (e.g.,cards 150, 160, 170, and 180) can be relevant to the context of thecurrent position in the on-demand program 120. For example, anindividual card can present guidance information relating to theon-demand program (e.g., program information, episode information,previous broadcast information, etc.), as shown in card 150. In anotherexample, an individual card can present text snippets relating to aproduct logo, face, and/or famous place that appears in the on-demandprogram (e.g., facts relating to a person appearing at that playbackposition in Episode 4 from Season 10 of “The Comedy Show”), as shown incard 160. Similarly, an individual card can present a portion of webcontent relating to a product logo, face, and/or famous place thatappears in the on-demand program (e.g., a portion of a webpage include abiography and images relating to a person appearing at that playbackposition in Episode 4 from Season 10 of “The Comedy Show”), as shown incard 170. In yet another example, as shown in card 180, an individualcard can present news articles, social media messages, or otherinformation relating to a topic being discussed at that playbackposition in Episode 4 from Season 10 of “The Comedy Show.”

It should be noted that the supplemental information in cards 150, 160,170, and/or 180 can be any suitable representation of supplementalinformation, such as a snippet of supplemental information (e.g., aquick fact or any other suitable text snippet), a thumbnail image, alink or uniform resource locator (URL) to additional content, anextracted keyword mentioned in subtitle information, an extracted topicdiscussed in the program, an identified song where at least a portion ofthe song was played back in the program, other songs from an artisthaving at least a portion of a song played back in the program, etc.

In some implementations, timing information 190 can be included with anindividual card (e.g., one of cards 150, 160, 170, and 180). Forexample, as shown in FIG. 1, card 180 can include timing information 190that indicates which specific content that card 180 refers or relatesto. More particularly, card 180 can include timing information 190 thatstates “Mentioned 5 minutes ago” or “5 minutes” such that the userviewing timing information 190 knows that the card may not pertain tosomething occurring exactly at the current time, but something thatoccurred recently. In another example, card 180 can include timinginformation that indicates how long card 180 will remain to bepresented. Upon the time displayed in indicator 190 elapsing, anothercard that presents information relevant to the context of the playbackposition of on-demand program 120 can be provided. Alternatively, card180 can be removed from page 100.

It should be noted that supplemental information in cards 150, 160, 170,and/or 180 can be presented in any suitable manner. For example, asshown in FIG. 1, the supplemental information in cards 150, 160, 170,and/or 180 can be provided within a window that overlays a video windowpresenting on-demand program 120. In another example, the supplementalinformation in card 150, 160, 170, and/or 180 can be provided as atransparency, where the supplemental information can be overlaid on thevideo window presenting on-demand program 120 without inhibiting theviewability of on-demand program 120. In yet another example, thesupplemental information in cards 150, 160, 170, and/or 180 can beprovided in a window positioned adjacent to the video window presentingon-demand program 120.

The presentation of supplemental information as cards can includepresentation of any suitable images, icons, graphics, videos,animations, sounds, haptic feedback, or any other suitable accompanyingcontent. For example, in some implementations, presentation of a cardcan include presenting an accompanying sound (e.g., a tone, a verbaldescription of the card content, or any other sounds) or accompanyinghaptic feedback (e.g., vibration of a user device). It should be notedthat, although there are four cards 150, 160, 170, and 180 shown in FIG.1, any suitable number of cards (including none) can be presented to auser.

In some implementations, in addition to presenting on-demand program120, page 100 can include any other suitable information. For example,page 100 can also include program information 130, such as programtitle, episode title, season number, episode number, original air date,channel information, channel identifier information, and/or detailedinformation associated with on-demand program 120. In another example,page 100 can also include comments provided by users that consumedon-demand program 120, an indication as to the number of viewers thatprovide a positive recommendation of on-demand program 120, links forsharing on-demand program 120 with other users, and/or any othersuitable social media-related information. In yet another example, page100 can include any suitable logo, such as a logo associated with theon-demand content provider or a logo associated with a channel that theprogram was originally aired. It should be noted that page 100 caninclude any suitable text, graphics, images, video, or any othersuitable content in accordance with some implementations. In someimplementations, information 130, such as a logo, can be omitted frompage 100.

Although not shown in FIG. 1, in some implementations, the supplementalcontent presented in cards can be interacted with by a user. Forexample, a card can be removed from page 100 if a user is not interestedor is no longer interested in the supplemental information presented onthe card. As another example, a card can be selected to perform anaction or present additional information (e.g., access a link to reviewan article relating to the context of the on-demand program). As a moreparticular example, if a card which displays a portion of a documentincluding actor information is presented, the card can be selected, andin response, an action can be performed, for example, launching a webbrowsing application that accesses the page with the document containingactor information. Cards can be interacted with in any suitable way. Forexample, in some implementations, cards can be dismissed by clicking ortapping on the card or on a “dismiss” icon (e.g., an “X” at the cornerof the card or any other suitable icon). As another particular example,in some implementations, cards can be dismissed by swiping or draggingthe card off the border of page 100. Similarly, cards can be selected byclicking, tapping, or any other suitable mechanism.

Turning to FIG. 2, an illustrative example 200 of a process forpresenting supplemental content is shown in accordance with someimplementations of the disclosed subject matter. At 210, a computingdevice can load a web browsing application that includes a web browserextension for presenting supplemental content. For example, thecomputing device can be a desktop computer, a laptop computer, acellular telephone, a smartphone, a tablet computing device, a wearablecomputing device, a gaming device, etc.

At 220, the web browsing application can be used to access a contentprovider website. The content provider website can be any suitableon-demand content provider, steaming media content provider, etc. Forexample, in response to accessing the content provider website, thecontent provider website can provide the user with an opportunity toselect one or more of the media content items for playback. It should benoted the media content items can include one or more programs fromvarious sources, such as on-demand programs, Internet content, streamingprograms, etc.

At 230, the web browsing application, using the web browser extension,can determine that a particular media content item has been selected forplayback. For example, in response to browsing through available mediacontent items on a content provider website, the web browsingapplication can determine that the user has selected a particularepisode of a particular season of a particular program for on-demandplayback on the content provider website. In a more particular example,the particular episode of the particular season of the particularon-demand program can be a program that was previously broadcastedover-the-air, broadcasted by a cable television provider, broadcasted bya telephone television provider, and/or broadcasted by a satellitetelevision provider.

In some implementations, the web browser extension executing on the webbrowsing application can enter a media content detection mode, which canbegin by processing the page that plays back the media content item onthe content provider website and obtaining media content informationcorresponding to the media content item being presented.

In some implementations, the media content information can include anaudio sample of the on-demand media content currently being presented.For example, the web browsing application can activate an audio inputdevice coupled to the computing device, such as a microphone, where theaudio input device captures and records the audio sample or any othersuitable audio data associated with a presented on-demand program. Inanother example, the web browsing application can receive a userindication to store audio data for transmission to an audio matchingservice.

In some implementations, the web browsing application can activate anaudio input device that captures audio from its surroundings and usesappropriates filters and/or other audio enhancements to extract an audiofingerprint or filtered audio data associated with the presentedon-demand program. For example, silent portions of the audio sample canbe removed to reduce the file size of the audio sample. In anotherexample, portions of the audio sample having distortion can be removedto reduce the file size of the audio sample. In yet another example, inresponse to determining that at least a portion of the audio sample doesnot include audio data having a given quality, the web browsingapplication can prompt the user of the web browsing application torecord another audio sample.

In some implementations, the web browsing application executing on thecomputing device can generate an audio fingerprint of the audio sampleusing any suitable audio fingerprinting technique at 230. The audiofingerprint can be a digital representation generated from the receivedaudio sample, where the audio fingerprint can be used to identify thesame or similar portions of audio data. Alternatively, the web browsingapplication can transmit the audio sample associated with the program toa search system or an audio matching system that generates an audiofingerprint.

In some implementations, the web browsing application can transmit theobtained audio sample and/or the audio fingerprint for analysis. Forexample, upon selecting a media content item for playback using the webbrowsing application and activating the audio input device, the webbrowsing application can automatically stream the audio sample and/orthe audio fingerprint to a search system that accesses a matchingservice (e.g., an audio fingerprint matching service). As describedabove, the web browsing application can stream the audio sample to asearch system, where the search system generates an audio fingerprintcorresponding to the audio sample and transmits the audio fingerprint toan audio fingerprint matching service for analysis.

Additionally or alternatively, the media content information can alsoinclude a video sample of the on-demand content currently beingpresented. For example, the web browsing application can capture a videosample or any other suitable video data associated with a presentedon-demand program. In another example, the web browsing application canreceive a user indication to store video data for transmission to avideo matching service.

In some implementations, the web browsing application executing on thecomputing device can generate a video fingerprint of the video sampleusing any suitable video fingerprinting technique. The video fingerprintcan be a digital representation generated from the received videosample, where the video fingerprint can be used to identify the same orsimilar portions of video data. Alternatively, the web browsingapplication can transmit the video sample associated with the program toa search system or a video matching system that generates a videofingerprint.

In some implementations, the web browsing application can transmit theobtained video sample and/or the video fingerprint for analysis. Forexample, upon selecting a media content item for playback using the webbrowsing application, the web browsing application can automaticallystream the video sample and/or the video fingerprint to a search systemthat accesses a matching service (e.g., a video fingerprint matchingservice). As described above, the web browsing application can streamthe video sample to a search system, where the search system generates avideo fingerprint corresponding to the video sample and transmits thevideo fingerprint to a video fingerprint matching service for analysis.

Additionally or alternatively, the media content information can alsoinclude one or more screenshots of the on-demand content currently beingpresented. For example, the web browsing application can capture ascreenshot associated with a presented on-demand program. In anotherexample, the web browsing application can receive a user indication tostore screenshots for transmission to an image matching service.

In some implementations, additional information can be transmitted alongwith the audio sample, the video sample, and/or the image sample, suchas client device information, user profile information, locationinformation, user preferences, timing information, and other informationthat can be used to generate a search query associated with the sampleof the on-demand program.

It should be noted that in implementations described herein in which theweb browsing application or any other suitable client application (orother mechanisms described herein) collects information about aparticular user, the user can be provided with an opportunity to controlwhether the application collects information about particular usersand/or how collected user information is used by the application.Examples of information about a user can include the user's interestsand identifying information of the user (e.g., a user profile, usercredentials, device identification, etc.). Additionally, certaininformation about the user can be stored locally (e.g., not shared),encrypted, and/or treated in one or more ways before it is stored toremove personally identifiable information. For example, the applicationcan store user preferences and/or user interests for a particular userwith an anonymous user identifier (e.g., a user identifier that is notassociated with the user's name, the user's username and/or password,the user's email address, etc.). Using these techniques, the user canhave control over what information is collected about the user and/orhow that information is used by the application.

Turning to FIG. 3, upon receiving an indication from the web browsingapplication or the web browser extension that a content provider websitehas been accessed and receiving the sample and/or other additionalinformation at 310, the search system can identify the media contentitem being provided by the content provider website on the web browsingapplication at 320.

Generally speaking, the search system can identify the media contentitem being provided by the content provider website by determining anassociated media content item that has been provided by one or moretelevision sources. For example, the search system can use the receivedmedia content information to associate the on-demand program beingprovided by the on-demand source to a program provided by a televisionsource. In a more particular example, the received media contentinformation associated with the on-demand program can be compared withmedia content information associated with television programs providedby one or more television providers (e.g., a cable television provider,a telephone television provider, a satellite television provider, etc.).

It should be noted that, in some implementations, prior to identifyingthe media content item, the search system can verify whether the contentprovider website is included on a list of particular content providerwebsites. For example, in response to determining that the particularcontent provider website accessed by a user using the web browsingapplication is not included on the list of particular content providerwebsites (e.g., not approved for providing supplemental content items),the search system can inhibit the processing of information from suchweb browsing applications. In another example, in response todetermining that the particular content provider website accessed by auser using the web browsing application is not included on the list ofparticular content provider websites (e.g., not approved for providingsupplemental content items), the search system can cause the web browserextension or any other suitable client application to inform the userusing the web browsing application that the presentation of supplementalcontent items is not supported.

In some implementations, a fingerprint of the received sample can begenerated. For example, an audio fingerprint of the received audiosample or a video fingerprint of the received video sample can begenerated. Again, the fingerprint can be generated by the web browsingapplication or any other suitable application and transmitted to thesearch system or a fingerprint matching system for analysis.

The search system can then compare the generated fingerprint to multiplefingerprints. For example, the generated audio fingerprint associatedwith the audio sample from the web browsing application can be comparedagainst a subset of stored audio fingerprints. In another example, thegenerated audio fingerprint associated with the audio sample from theon-demand program can be compared against live television program audiofingerprints that are stored and indexed by program in a database. In amore particular example, the search system can access a database orother suitable storage device containing multiple audio fingerprints andfilter the multiple audio fingerprints to a subset of audio fingerprintsbased on location information, user preferences, and other informationreceived from the web browsing application. In this example, only audiofingerprints associated with the received location information can beused to compare with the generated audio fingerprint. In anotherexample, particular audio fingerprints from the database or storagedevice can be used for comparison before others, such as audiofingerprints associated with popular channels, newer audio fingerprints,audio fingerprints commonly matched, etc.

In some implementations, the search system can, for example, includecapture modules that receive and process signals from multiple sources.These capture modules can, for each channel, capture video screenshotsat particular time intervals (e.g., every two or three seconds) and/orgenerate audio fingerprints from audio data at particular timeintervals. Generally speaking, these capture modules can monitor mediacontent from multiple content sources and generate video screenshots,audio fingerprints, and/or any other suitable content identifier. Moreparticularly, these capture modules can store the generated videoscreenshots, audio fingerprints, and other content identifiers in astorage device. For example, a capture module can monitor channelsproviding broadcast television content and store generated audiofingerprints in a database that is indexed by channel and time.

It should be noted that, in some implementations, generated videoscreenshots, audio fingerprints, and other content identifiers indexedby channel and time in a database can be discarded after a predeterminedtime. For example, in response to determining that a video screenshot isassociated with a time outside of a particular time window, the searchsystem or any other suitable component can remove the video screenshotfrom the database.

In some implementations, it can be determined whether a match is found.If no matching audio fingerprint is found, the search system can wait toreceive additional audio samples from the web browsing application.Additionally or alternatively, the search system can transmit a messageto the web browsing application requesting that the user restart the webbrowsing application, transmit another audio sample, and/or input orverify particular information (e.g., program information, such aschannel number, service provider information, program title, etc.).

In response to identifying a fingerprint match, the search system canidentify a media content item or any other suitable content identifierassociated with the matched fingerprint. For example, a matching audiofingerprint can be associated with a particular episode of a particularprogram (e.g., Episode 4 from Season 10 of “The Comedy Show” program inFIG. 1) and the search system can use this association to determine thatthe user of web browsing application is watching a television programthat was previously aired on a channel by a television provider. Inanother example, the search system can transmit a notification to theweb browsing application that the media content item has beenidentified. This notification can include, for example, an indicationthat the search system has successfully identified the media contentitem from the received audio sample. In some implementations, inresponse to the user of the web browsing application determining thatthe identified media content item is incorrect, the web browsingapplication can provide the user with an opportunity to indicate thecorrect media content item or transmit another audio, video, and/orimage sample.

It should be noted that, in some implementations, the web browsingapplication can initiate the media content detection mode withoutcapturing audio information, video information, image information, orusing fingerprints (e.g., step 230 of FIG. 2 and step 320 of FIG. 3).For example, in response to detecting that the user has selected anon-demand media content item for playback on a content provider website,the web browsing application can obtain media content informationassociated with the on-demand media content item, such as program title,episode number, episode title, series number, summary of content,creator information, publisher information, original air date, durationinformation, etc. In response to obtaining such media contentinformation and transmitting the media content information to the searchsystem, the search system can determine whether one or more pieces ofthe media content information match a program that has been broadcast bya television provider. For example, the episode number and the programtitle can be used to associate the on-demand program with a particularepisode of a television program that was broadcasted by a televisionprovider. In another example, the search system can obtain the mediacontent information by analyzing the video player on the contentprovider website that is playing back the media content item.

Additionally or alternatively, the search system can identify the mediacontent item being provided by the content provider website bydetermining that the media content item matches a media content itemfrom any suitable media database. For example, the media content itemsstored in a media database can be processed and one or more fingerprintscan be generated from the stored media content items. The search systemcan determine whether a fingerprint of the sample received from the webbrowsing application matches a fingerprint of a media content item in asuitable media database. In another example, the search system cantransmit a query to an internal media database that includes thefingerprint or any other suitable sample of the content provided on theweb browsing application, where the internal media database can respondwith information that identifies the media content item.

In response to identifying the media content item and determining thatthe on-demand media content item is associated with a media content itemthat was previously broadcasted by a television provider, the searchsystem can determine whether supplemental content corresponding to thepreviously broadcasted media content item is available at 330.

As described above, the search system can, for example, include capturemodules that receive and process signals from multiple sources (e.g.,multiple television channels including television programs that arebroadcasted by one or more television providers). These capture modulescan, for each channel, process the provided television programs. Forexample, in response to receiving frames (e.g., frames of livetelevision program), these capture modules can process the frames todetermine whether at least one of the frames includes an object ofinterest using one or more of: an object recognition technique toidentify objects, landmarks, places, etc. that are shown in a frame, afacial recognition technique to identify faces of actors, actresses,and/or characters that appear in a frame, and/or optical characterrecognition to identify text portions, such as a telephone number, awebsite address, a business name, a product name, etc. In anotherexample, these capture modules can extract keywords and/or entityinformation that is included in subtitle information corresponding tothe program. In yet another example, these capture modules can determinetopics that are being discussed in a program (e.g., by processing audiodata and/or an audio fingerprint). In a further example, these capturemodules can identify a song that is played in a program (e.g., byprocessing audio data and/or an audio fingerprint). Upon performing thisanalysis, the object of interest can be associated with a content item.For example, the content item can include a selectable link that can beassociated with an identified text portion that includes a websiteaddress or a business name. In another example, the content item can bea selectable link or a text snippet corresponding to a news article, webcontent, or any other suitable content that discusses an extractedkeyword or an identified topic. In yet another example, the content itemcan be a selectable link directed to a music provider for playing and/orpurchasing the identified song in the program.

In some implementations, these capture modules can store the contentitems as supplemental content items in a storage device. For example, acapture module can monitor channels providing broadcast televisioncontent and store supplemental content in a database that is indexed byprogram and time. In a more particular example, a capture module canstore supplemental content along with timestamped audio and imagefingerprints for every N millisecond in a database while a program isbeing broadcasted by a television provider.

Referring back to 330, in response to determining whether supplementalcontent corresponding to the media content item is available, the searchsystem can determine whether supplemental content corresponding to thepreviously broadcasted media content item is available and obtainsupplemental content from such a database. For example, in response todetermining that the particular episode of “The Comedy Show” shown inFIG. 1 is associated with a previously broadcasted television programand that supplemental content was generated by one or more capturemodules, the search system can access the portion of the databasecorresponding to the particular episode of “The Comedy Show” and obtainthe supplemental content. In a more particular example, the supplementalcontent corresponding to the particular episode of the program can betimestamped to relate to particular portions of the program.

Alternatively, the search system can instruct the web browser extensionor any other suitable client application to obtain supplemental contentcorresponding to the broadcasted media content item from one or moreservers (e.g., capture servers). In response, the supplemental contentcan be locally stored in a suitable storage device associated with thecomputing device.

Additionally or alternatively, the search system can process mediacontent items stored in an internal media database such thatsupplemental information is extracted from the media content items. Suchsupplemental information can be presented to a user viewing an on-demandvideo of the same content using a web browsing application.

At 340, the search system can cause the web browser extension or anyother suitable client application to present an indicator thatsupplemental content is available. An example of the availabilityindicator (e.g., availability indicator 140) is shown, for example, inFIG. 1. Referring back to FIG. 2, in response to presenting an indicatorthat supplemental content associated with the selected media contentitem is available at 240, the web browsing application can cause themedia content item that is provided by the content provider website tobe presented at 250. For example, the web browsing application can beingplayback of the selected media content item.

Upon presenting the media content item, the web browsing application candetect whether a user has selected the availability indicator at 260.For example, the web browsing application can transmit an indicationthat the user has selected the availability indicator to the searchsystem. In response, the web browsing application can receive andpresent one or more interfaces including supplemental content relatingto the presented media content item at 270. For example, as shown inFIG. 1, cards 150, 160, 170, and/or 180 including supplementalinformation can be presented to the user while viewing the selectedmedia content item.

It should be noted that, in some implementations, the web browsingapplication can provide a user-selectable option for presentingsupplemental content upon determining that such supplemental content isavailable. For example, in response to determining that supplementalcontent for a television program that was broadcasted by a televisionprovider corresponding to the selected on-demand program is available,the web browsing application can, using the web browser extension,automatically present the supplemental content concurrently whilepresenting the on-demand program. In a more particular example, the webbrowsing application can, using the web browser extension, automaticallyupdate the supplemental content to be contextually relevant to contentpresented in the on-demand program.

Referring back to FIG. 3, in some implementations, the search system canenter a synchronization mode upon determining that supplemental contentis available for the on-demand program.

At 350, in response to causing an indicator that supplemental content isavailable to be presented and obtaining such supplemental content, thesearch system can enter a synchronization mode that awaits the user tobegin playback of the selected on-demand media content item (e.g., theon-demand episode of “The Comedy Show in FIG. 1). For example, thesearch system can determine a particular playback point in the episodeof the on-demand program that is being presented to the user. In a moreparticular example, as described above, the web browsing application cantransmit media content information to identify the particular playbackpoint in the on-demand program.

In some implementations, the media content information can include anaudio sample of the on-demand media content currently being presented.For example, the web browsing application can activate an audio inputdevice coupled to the computing device, such as a microphone, where theaudio input device captures and records the audio sample or any othersuitable audio data associated with a presented on-demand program. Inanother example, the web browsing application can receive a userindication to store audio data at a particular point in the presentationof the on-demand media content item for transmission to an audiomatching service.

In some implementations, the web browsing application executing on thecomputing device can generate an audio fingerprint of the audio sampleusing any suitable audio fingerprinting technique. The audio fingerprintcan be a digital representation generated from the received audiosample, where the audio fingerprint can be used to identify the same orsimilar portions of audio data. Alternatively, the web browsingapplication can transmit the audio sample associated with the program toa search system or an audio matching system that generates an audiofingerprint.

In some implementations, the web browsing application can transmit theobtained audio sample and/or the audio fingerprint for analysis. Forexample, the web browsing application can transmit the audio sampleand/or the audio fingerprint to a search system that accesses a matchingservice (e.g., an audio fingerprint matching service). As describedabove, the web browsing application can stream the audio sample to asearch system, where the search system generates an audio fingerprintcorresponding to the audio sample and transmits the audio fingerprint toan audio fingerprint matching service for analysis.

Additionally or alternatively, the media content information can alsoinclude a video sample of the on-demand content currently beingpresented. For example, the web browsing application can capture a videosample or any other suitable video data associated with a presentedon-demand program. In another example, the web browsing application canreceive a user indication to store video data at a particular point inthe presentation of the on-demand media content item for transmission toa video matching service.

In some implementations, the web browsing application executing on thecomputing device can generate a video fingerprint of the video sampleusing any suitable video fingerprinting technique. The video fingerprintcan be a digital representation generated from the received videosample, where the video fingerprint can be used to identify the same orsimilar portions of video data. Alternatively, the web browsingapplication can transmit the video sample associated with the program toa search system or a video matching system that generates a videofingerprint.

In some implementations, the web browsing application can transmit theobtained video sample and/or the video fingerprint for analysis. Forexample, the web browsing application can transmit the video sampleand/or the video fingerprint to a search system that accesses a matchingservice (e.g., a video fingerprint matching service). As describedabove, the web browsing application can stream the video sample to asearch system, where the search system generates a video fingerprintcorresponding to the video sample and transmits the video fingerprint toa video fingerprint matching service for analysis.

Additionally or alternatively, the media content information can alsoinclude one or more screenshots of the on-demand content currently beingpresented. For example, the web browsing application can capture ascreenshot associated with a presented on-demand program. In anotherexample, the web browsing application can receive a user indication tostore screenshots for transmission to an image matching service.

Upon receiving an audio fingerprint, a video fingerprint, a screenshot,or any suitable media content information, the search system can thencompare the generated fingerprint to multiple fingerprints. For example,the generated audio fingerprint associated with the audio sample fromthe web browsing application can be compared against a subset of storedaudio fingerprints. In another example, the generated audio fingerprintassociated with the audio sample from the on-demand program can becompared against live television program audio fingerprints that arestored and indexed by program in a database. Similarly, the generatedvideo fingerprint associated with the video sample from the web browsingapplication can be compared against a subset of stored videofingerprints captured from broadcasted television programs. In yetanother example, a screenshot of the on-demand program from the webbrowsing application can be compared against live television programscreenshots that are stored and indexed by program in a database. Inresponse to determining that the received audio fingerprint, videofingerprint, and/or screenshot matches information stored in one or moredatabases, the search system can obtain information from the matchingfingerprint and determine the playback position or time offset of theon-demand program. For example, particular media information associatedwith a fingerprint of a previously broadcasted television program canprovide timing information, such as a time frame (e.g., between 0:12minutes and 0:13 minutes), a time offset (e.g., 10 minutes and 3 secondsfrom the start of the program), etc.

In some implementations, a video sample, a screenshot, or a videofingerprint can include a portion of the video player provided by thecontent provider website. In this implementation, the search system candetermine the playback position by processing the image data of thevideo player. For example, the search system can analyze the image dataof the video player to extract timing information from the video player.Alternatively, the search system can receive information relating to thevideo player on the content provider website that is playing back theon-demand media content item. In response, the search system can extractplayback position or any other suitable timing information from suchinformation.

In some implementations, the search system can transmit a query to thevideo player provided by the content provider website that requests theplayback position or any other suitable timing information. In responseto the query, the video player or any other suitable portion of thecontent provider website can provide a response message that includesthe current playback position, the time offset, or any other suitabletiming parameter.

Referring back to FIG. 3, in response to determining a particularplayback position or other suitable timing information at 350, thesearch system can select suitable portions of supplemental informationfor presentation at 360 and transmit the supplemental information to theweb browser extension at 370. For example, timing information or anyother suitable synchronization data can be used to determine aparticular playback position of the on-demand program and then determinewhich piece of supplemental content from the associated broadcastedprogram relates to that particular playback position. In a moreparticular example, as shown in FIG. 1, in response to determining thatthe episode of “The Comedy Show” is ten minutes into the program, thesearch system can obtain cards 150, 160, 170, and/or 180 that includesupplemental information that is contextually relevant to thatparticular playback position of the program. The guest speaker in theon-demand program may be “John Smith” and the topic of conversation inthe on-demand program may be “hydraulic fracking.” In response, cards160 and 170 that provide text snippets relating to “John Smith” and card180 that provides social media snippets relating to the topic “hydraulicfracking” can be presented to the user of the web browsing application.

In some implementations, the search system can obtain all of thesupplemental content items associated with that particular playbackposition for that particular episode of the previously broadcastedtelevision program corresponding to the on-demand program currentlybeing played back. In a more particular implementation, the searchsystem can query one or more databases that contain supplemental contentitems for programs broadcasted by a television provider, where the queryincludes particular timing information, episode information, and/orprogram information. In response, the search system can receive links tocorresponding supplemental content items. The search system can thentransmit the supplemental content items to the web browsing applicationon the computing device for presentation. Alternatively, the searchsystem can select a subset of the supplemental content items forpresentation for a particular period of time and, in response to thatperiod of time elapsing, select another subset of the supplementalcontent items for presentation. It should be noted that, in someimplementations, selection of the number of supplemental content itemscan depend on multiple criterion, such as device type (e.g., desktopcomputing device, mobile device, etc.), display size (e.g., LCD displaydevice, tablet computing device, mobile device, etc.), networkconnection, etc.

It should be noted that, in some implementations, the web browsingapplication can continuously and/or periodically transmit an audiosample, a video sample, a screenshot, a fingerprint, or any othersuitable media data relating to an on-demand program being presented tothe search system. In response, the web browsing application can beprovided with updated supplemental content for presentation and/orselection.

Turning to FIG. 4, an illustrative example 400 of a generalizedschematic diagram of a system on which the mechanisms for presentingcontextual information corresponding to on-demand media content is shownin accordance with some implementations of the disclosed subject matter.As illustrated, system 400 can include one or more computing devices410. Computing devices 410 can be connected by one or more communicationlinks 408 to a communications network 406 that can be linked via acommunications link 404 to a server 402. In some implementations,computing devices 410 and server 402 can be linked via a communicationslink 412 to one or more content sources 414.

System 400 can include one or more servers 402. Server 402 can be anysuitable server for providing access to the application, such as aprocessor, a computer, a data processing device, or any suitablecombination of such devices. For example, the application can bedistributed into multiple backend components and multiple frontendcomponents and/or interfaces. In a more particular example, backendcomponents, such as data collection and data distribution can beperformed on one or more servers 402.

In some implementations, front-end components of the application (or aportion of the application), such as a user interface and/or channelidentification features can be performed on one or more computingdevices 410.

In some implementations, each of the computing devices 410 and server402 can be any of a general purpose device such as a computer or aspecial purpose device such as a client, a server, etc. Any of thesegeneral or special purpose devices can include any suitable componentssuch as a hardware processor (which can be a microprocessor, digitalsignal processor, a controller, etc.), memory, communication interfaces,display controllers, input devices, etc. For example, computing device410 can be implemented as a smartphone, a tablet computer, a wearablecomputer, a smart television, a set-top box, a digital media receiver, agame console, a personal computer, a laptop computer, a personal dataassistant (PDA), a home entertainment system, any other suitablecomputing device, or any suitable combination thereof. In this example,computing device 410 can present a program, such as an on-demandprogram, and supplemental content that may be contextually relevant tothe program (e.g., via a web browser extension executing on a webbrowsing application installed on computing device 410).

For example, in some implementations, a program can be presented using afirst computing device 410, such as a smart television, a set-top box, adigital media receiver, a desktop computer, etc., and supplementalcontent can be presented using a second computing device 410, such as atablet computer, a smartphone, a wearable computer, a PDA, etc.

In some implementations, content source(s) 414 can be any suitablecontent sources, such as a cable (or telephone) television contentsource, a satellite television content source, an on-demand programcontent source, an over-the-top program content source, an Internetcontent source, a streaming program content source, any other suitablecontent source, or any suitable combination thereof.

In some implementations, communications network 406 can be any suitablecomputer network or combination of such networks including the Internet,an intranet, a wide-area network (WAN), a local-area network (LAN), awireless network, a digital subscriber line (DSL) network, a frame relaynetwork, an asynchronous transfer mode (ATM) network, a virtual privatenetwork (VPN), etc. Communications links 404, 408, and 412 can be anycommunications links suitable for communicating data among computingdevices 410, server 402, and/or content source(s) 414, such as networklinks, dial-up links, wireless links, hard-wired links, any othersuitable communications links, or any suitable combination of suchlinks. Computing devices 410 can enable use of the techniques describedherein that can allow the features of the mechanisms to be used.Computing devices 410, server 402 and content source(s) 414 can belocated at any suitable location.

In some implementations, server 402 can include one or more modules420-430 and/or databases 440-442 that can be interconnected, forperforming various functions of the application and/or facilitating themechanisms for presenting supplemental information corresponding toon-demand media content.

In some implementations, a front-end server module 420 can performprocesses for presenting supplemental information as described above inconnection with, for example, FIGS. 1-3. For example, the front-endserver can act as a proxy between the web browsing application and/orthe web browser extension executing on computing device 410 and otherserver modules, such as capture module 422, matching module 424, andsupplemental information module 426.

In some implementations, a capture module 422 can receive media datarelated to a program or a channel, such as video data, audio data,electronic program guide data, metadata, subtitles or captioningcontent, etc., as described above in connection with, for example, FIGS.1-3. Additionally or alternatively, capture module 422 can extractvarious media data from content provided from content sources asdescribed in connection with, for example, FIGS. 1-3. Such extractedmedia data can include, for example, audio fingerprints, subtitles, etc.This information can be stored, for example, in a database (not shown)for use by the search application executing on front-end server 420 inidentifying media content items, obtaining supplemental content items,and/or various other operations.

In some implementations, guidance data 440, such as an electronicprogram guide (EPG) database and a closed caption database, can beaccessed. For example, front-end server module 420 can access guidancedata 440 to obtain program information related to an identified programand use the program information to search for matching supplementalcontent items. It should be noted that, in some implementations,guidance data 440 can be provided from, for example, content source(s)414.

In some implementations, a matching module 424 can receive audiofingerprint data and/or video fingerprint data for a program fromcapture module 422 and compare the received fingerprint data againststored fingerprint data in a fingerprint database 442. This fingerprintdatabase can be used, for example, to determine a program that is beingpresented to a user, to determine a particular playback position in theprogram, etc.

In some implementations, a supplemental content module 426 can findmatching supplemental content items associated with a program usingkeyword extraction module 430 and synchronization module 428. Forexample, keyword extraction module 430 can extract keywords fromcaptured audio data, video data, and/or subtitle information and obtainsupplemental content items from multiple content sources (e.g., contentsources 414). In some implementations, keyword extraction module 430 canthen rank each of the supplemental content items by recency, popularity(e.g., popularity based on click-through-rate), etc. In response toranking the supplemental content items, keyword extraction module 430can select at least one of the supplemental content items for presentingto the user at a particular position in the program.

In some implementations, a timeline can be associated with each program,where a supplemental content item can be associated with a timestamp ortime period indication that identifies the time that the correspondingmedia content occurred (e.g., portion of the television program thatprovides the text snippet).

In response to identifying the on-demand content item and determining anassociated program that has been broadcasted by a television provider,front-end server 420 can access the timeline associated with theassociated program and return supplemental content (if any) that can bepresented to the user. In some implementations, a representation of theone or more supplemental content items can then be provided. Forexample, the representation can include a snippet of web content, athumbnail image, a URL to a document, etc.

It should be noted that, although modules 420-430 and databases 440-442are shown as being included in server 402, these modules and/ordatabases can be provided in different servers in various combinations.For example, supplemental information module 426 can be provided on afirst server and modules 420-424 can be provided on a second server. Asanother example, each module can be provided on a different server. Itshould be noted that these are merely examples, and the modulesdescribed can be organized in any suitable manner.

It should also be noted that modules 420-430 can be included as part ofthe application, can each be a separate application, or can be part ofany other suitable application.

FIG. 5 illustrates an example 500 of hardware that can be used toimplement server 402 and one of computing devices 410 depicted in FIG. 4in accordance with some implementations of the disclosed subject matter.Referring to FIG. 5, computing device 410 can include a hardwareprocessor 512, a display 514, an input device 516, and memory 518, whichcan be interconnected. In some implementations, memory 518 can include astorage device (such as a non-transitive computer-readable medium) forstoring a computer program for controlling hardware processor 512.

Hardware processor 512 can use the computer program to present ondisplay 514 content and/or an interface that allows a user to interactwith the web browsing application and to send and receive data throughcommunications link 408. It should also be noted that data receivedthrough communications link 408 or any other communications links can bereceived from any suitable source. In some implementations, hardwareprocessor 512 can send and receive data through communications link 408or any other communication links using, for example, a transmitter,receiver, transmitter/receiver, transceiver, or any other suitablecommunication device. Input device 516 can be a computer keyboard, amouse, a trackball, a keypad, a remote control, any other suitable inputdevice, or any suitable combination thereof. Additionally oralternatively, input device 516 can include a touch screen display 514that can receive input (e.g. using a finger, a stylus, or the like).

Server 402 can include a hardware processor 522, a display 524, an inputdevice 526, and memory 528, which can be interconnected. In someimplementations, memory 528 can include a storage device for storingdata received through communications link 404 or through other links,and processor 522 can receive commands and values transmitted by one ormore users of, for example, computing device 410. The storage device canfurther include a server program for controlling hardware processor 522.

The mechanisms described herein for providing news items and othernews-related content based on media content can be implemented incomputing devices 410 and/or server 502 as software, firmware, hardware,or any suitable combination thereof.

In some implementations, server 402 can be implemented as one server orcan be distributed as any suitable number of servers. For example,multiple servers 402 can be implemented in various locations to increasereliability, function of the application, and/or the speed at which theserver can communicate with computing devices 410.

In some implementations, the client application can include anapplication program interface (not shown), and/or can be resident inmemory 518 of computing device 410 and/or memory 528 of server 402.Additionally or alternatively, a graphical user interface (“GUI”) can bedistributed to computing device 510, which can allow a user to interactwith the client application resident at, for example, server 402.

In some implementations, the application can include client-sidesoftware, server-side software, hardware, firmware, or any suitablecombination thereof. For example, the application can encompass acomputer program that causes one or more processors to execute thecontent generation application. As another example, the application(s)can encompass a computer program written in a programming languagerecognizable by computing device 510 and/or server 502 that is executingthe application(s) (e.g., a program written in a programming language,such as, Java, C, Objective-C, C++, C #, Javascript, Visual Basic, HTML,XML, ColdFusion, any other suitable approaches, or any suitablecombination thereof).

In some implementations, the application can encompass one or moreWeb-pages or Web-page portions (e.g., via any suitable encoding, such asHyperText Markup Language (“HTML”), Dynamic HyperText Markup Language(“DHTML”), Extensible Markup Language (“XML”), JavaServer Pages (“JSP”),Active Server Pages (“ASP”), Cold Fusion, or any other suitableapproaches).

Accordingly, methods, systems, and media for presenting supplementalcontent corresponding to media content are provided.

In some implementations, any suitable computer readable media can beused for storing instructions for performing the functions and/orprocesses described herein. For example, in some implementations,computer readable media can be transitory or non-transitory. Forexample, non-transitory computer readable media can include media suchas magnetic media (such as hard disks, floppy disks, etc.), opticalmedia (such as compact discs, digital video discs, Blu-ray discs, etc.),semiconductor media (such as flash memory, electrically programmableread only memory (EPROM), electrically erasable programmable read onlymemory (EEPROM), etc.), any suitable media that is not fleeting ordevoid of any semblance of permanence during transmission, and/or anysuitable tangible media. As another example, transitory computerreadable media can include signals on networks, in wires, conductors,optical fibers, circuits, any suitable media that is fleeting and devoidof any semblance of permanence during transmission, and/or any suitableintangible media.

It should be understood that the above described steps of the processesof FIGS. 2 and 3 can be executed or performed in any order or sequencenot limited to the order and sequence shown and described in thefigures. Also, some of the above steps of the processes of FIGS. 2 and 3can be executed or performed substantially simultaneously whereappropriate or in parallel to reduce latency and processing times.

It should also be noted that, as used herein, the term mechanism canencompass hardware, software, firmware, or any suitable combinationthereof.

Although the invention has been described and illustrated in theforegoing illustrative implementations, it is understood that thepresent disclosure has been made only by way of example, and thatnumerous changes in the details of implementation of the invention canbe made without departing from the spirit and scope of the invention,which is limited only by the claims that follow. Features of thedisclosed implementations can be combined and rearranged in variousways.

What is claimed is:
 1. A method for providing information relating topresented media content, the method comprising: determining, using ahardware processor, whether a media content item being presented in avideo window corresponds with one or more media content items stored ina media database, wherein portions of image data corresponding to mediacontent items are each associated with a time; determining, using thehardware processor, that supplemental content has been generated for themedia content item being presented in the video window, wherein thesupplemental content is available for one or more of the portions ofimage data; selecting, using the hardware processor, a portion of thesupplemental content based on at least one keyword that was extractedfrom a portion of the image data that includes the video window playingback the media content item; causing, using the hardware processor, anoverlay interface that includes the portion of supplemental content tobe presented concurrently with the media content item being presented inthe video window.
 2. The method of claim 1, wherein the media contentitem is being presented in the video window of a web browsingapplication.
 3. The method of claim 2, further comprising receiving anindication from the web browsing application that the media content itemis being presented in the video window.
 4. The method of claim 3,wherein the indication includes at least one keyword that has beenextracted from the media content item being presented in the videowindow.
 5. The method of claim 2, wherein the web browser applicationincludes a web browser extension that presents an indicator within anaddress bar of the web browsing application that the supplementalcontent which is synchronized with the presentation of the media contentbeing presented in the video window is available.
 6. The method of claim5, wherein the web browsing application presents the overlay interfacethat includes the portion of supplemental content concurrently with themedia content item being presented in the video window in response todetecting that the indicator has been selected.
 7. The method of claim1, wherein the media content item being presented in the video window isdetermined to correspond with the one or more media content items storedin the media database further comprises: generating a fingerprint fromat least a portion of media information associated with the mediacontent item being presented in the video window; and determiningwhether the fingerprint matches one of a plurality of fingerprintsassociated with the one or more media content items from a fingerprintdatabase.
 8. The method of claim 1, further comprising determining thatthe media content item is an on-demand video that has been selected forpresentation in the video window by comparing the image datacorresponding to the media content item with an on-demand videodatabase.
 9. The method of claim 1, wherein determining whether themedia content item being presented in the video window corresponds withthe one or more media content items stored in the media database furthercomprises: extracting the media information from the media content itembeing presented in the video window.
 10. A system for providinginformation relating to media content, the system comprising: a hardwareprocessor that is configured to: determine whether a media content itembeing presented in a video window corresponds with one or more mediacontent items stored in a media database, wherein portions of image datacorresponding to media content items are each associated with a time;determine that supplemental content has been generated for the mediacontent item being presented in the video window, wherein thesupplemental content is available for one or more of the portions ofimage data; select a portion of the supplemental content based on atleast one keyword that was extracted from a portion of the image datathat includes the video window playing back the media content item;cause an overlay interface that includes the portion of supplementalcontent to be presented concurrently with the media content item beingpresented in the video window.
 11. The system of claim 10, wherein themedia content item is being presented in the video window of a webbrowsing application.
 12. The system of claim 11, wherein the hardwareprocessor is further configured to receive an indication from the webbrowsing application that the media content item is being presented inthe video window.
 13. The system of claim 12, wherein the indicationincludes at least one keyword that has been extracted from the mediacontent item being presented in the video window.
 14. The system ofclaim 11, wherein the web browser application includes a web browserextension that presents an indicator within an address bar of the webbrowsing application that the supplemental content which is synchronizedwith the presentation of the media content being presented in the videowindow is available.
 15. The system of claim 14, wherein the webbrowsing application presents the overlay interface that includes theportion of supplemental content concurrently with the media content itembeing presented in the video window in response to detecting that theindicator has been selected.
 16. The system of claim 10, wherein themedia content item being presented in the video window is determined tocorrespond with the one or more media content items stored in the mediadatabase further comprises: generating a fingerprint from at least aportion of media information associated with the media content itembeing presented in the video window; and determining whether thefingerprint matches one of a plurality of fingerprints associated withthe one or more media content items from a fingerprint database.
 17. Thesystem of claim 10, wherein the hardware processor is further configuredto determine that the media content item is an on-demand video that hasbeen selected for presentation in the video window by comparing theimage data corresponding to the media content item with an on-demandvideo database.
 18. The system of claim 10, wherein determining whetherthe media content item being presented in the video window correspondswith the one or more media content items stored in the media databasefurther comprises: extracting the media information from the mediacontent item being presented in the video window.
 19. A non-transitorycomputer-readable medium containing computer executable instructionsthat, when executed by a processor, cause the processor to perform amethod for providing information relating to media content, the methodcomprising: determining whether a media content item being presented ina video window corresponds with one or more media content items storedin a media database, wherein portions of image data corresponding tomedia content items are each associated with a time; determining thatsupplemental content has been generated for the media content item beingpresented in the video window, wherein the supplemental content isavailable for one or more of the portions of image data; selecting aportion of the supplemental content based on at least one keyword thatwas extracted from a portion of the image data that includes the videowindow playing back the media content item; causing an overlay interfacethat includes the portion of supplemental content to be presentedconcurrently with the media content item being presented in the videowindow.